Unified three dimensional virtual craniofacial and dentition model and uses thereof

ABSTRACT

A method and apparatus are disclosed enabling an orthodontist or a user to create an unified three dimensional virtual craniofacial and dentition model of actual, as-is static and functional anatomy of a patient, from data representing facial bone structure; upper jaw and lower jaw; facial soft tissue; teeth including crowns and roots; information of the position of the roots relative to each other; and relative to the facial bone structure of the patient; obtained by scanning as-is anatomy of craniofacial and dentition structures of the patient with a volume scanning device; and data representing three dimensional virtual models of the patient&#39;s upper and lower gingiva, obtained from scanning the patient&#39;s upper and lower gingiva either (a) with a volume scanning device, or (a) with a surface scanning device. Such craniofacial and dentition models of the patient can be used in optimally planning treatment of a patient.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional application of application Ser. No.13/107,913, filed May 15, 2011, pending, which is a continuation-in-partof application Ser. No. 12/772,208, filed May 1, 2010, pending. Thisapplication is related to two other divisional applications of Ser. No.13/107,913, Ser. Nos. 14/587,989 and 14/588,100, both filed on Dec. 31,2014 and are pending.

The subject matter of this application is related to the subject matterof the following applications. Priority to the related applications isnot claimed under 35 U.S.C. §120.

application Ser. No. 09/834,593, filed Apr. 13, 2001, now issued as U.S.Pat. No. 7,068,825;

application Ser. No. 09/835,007, filed Apr. 13, 2001, now issued as U.S.Pat. No. 7,027,642;

application Ser. No. 09/834,413, filed Apr. 13, 2001, now issued as U.S.Pat. No. 7,080,979;

application Ser. No. 09/835,039, filed Apr. 13, 2001, now issued as U.S.Pat. No. 6,648,640;

application Ser. No. 09/834,593, filed Apr. 13, 2001, now issued as U.S.Pat. No. 7,068,825;

application Ser. No. 10/429,123, filed May 2, 2003, now issued as U.S.Pat. No. 7,234,937; and

application Ser. No. 10/428,461, filed May 2, 2003, pending, which is acontinuation-in-part of application Ser. No. 09/834,412, filed Apr. 13,2001, now issued as U.S. Pat. No. 6,632,089.

The entire contents of each of the above listed patent application areincorporated by reference herein.

BACKGROUND OF THE INVENTION

A. Field of the Invention

This invention relates generally to the field of orthodontics. Moreparticularly, the invention relates to generating a three dimensionalunified virtual model of the craniofacial and dentition of a patientfrom volume scan and surface scan digital data; and using such model orportions thereof for planning orthodontic treatment of the patient,including surgery.

B. Description of Related Art

In orthodontics, a patient suffering from a malocclusion is typicallytreated by bonding brackets to the surface of the patient's teeth. Thebrackets have slots for receiving an archwire. The bracket-archwireinteraction governs forces applied to the teeth and defines the desireddirection of tooth movement. Typically, the bends in the wire are mademanually by the orthodontist. During the course of treatment, themovement of the teeth is monitored. Corrections to the bracket positionand/or wire shape are made manually by the orthodontist.

The key to efficiency in treatment and maximum quality in results is arealistic simulation of the treatment process. Today's orthodontistshave the possibility of taking plaster models of the upper and lowerjaw, cutting the model into single tooth models and sticking these toothmodels into a wax bed, lining them up in the desired position, theso-called set-up. This approach allows for reaching a perfect occlusionwithout any guessing. The next step is to bond a bracket at every toothmodel. This would tell the orthodontist the geometry of the wire to runthrough the bracket slots to receive exactly this result. The next stepinvolves the transfer of the bracket position to the originalmalocclusion model. To make sure that the brackets will be bonded atexactly this position at the real patient's teeth, small templates forevery tooth would have to be fabricated that fit over the bracket and arelevant part of the tooth and allow for reliable placement of thebracket on the patient's teeth. To increase efficiency of the bondingprocess, another option would be to place each single bracket onto amodel of the malocclusion and then fabricate one single transfer trayper jaw that covers all brackets and relevant portions of every tooth.Using such a transfer tray guarantees a very quick and yet precisebonding using indirect bonding.

However, it is obvious that such an approach requires an extreme amountof time and labor and thus is too costly, and this is the reason why itis not practiced widely. The normal orthodontist does not fabricateset-ups; he places the brackets directly on the patient's teeth to thebest of his knowledge, uses an off-the-shelf wire and hopes for thebest. There is no way to confirm whether the brackets are placedcorrectly; and misplacement of the bracket will change the directionand/or magnitude of the forces imparted on the teeth. While at thebeginning of treatment things generally run well as all teeth start tomove at least into the right direction, at the end of treatment a lot oftime is lost by adaptations and corrections required due to the factthat the end result has not been properly planned at any point of time.For the orthodontist this is still preferable over the lab processdescribed above, as the efforts for the lab process would still exceedthe efforts that he has to put in during treatment. And the patient hasno choice and does not know that treatment time could be significantlyreduced if proper planning was done.

U.S. Pat. No. 5,431,562 to Andreiko et al. describes a computerized,appliance-driven approach to orthodontics. In this method, first certainshape information of teeth is acquired. A uniplanar target archform iscalculated from the shape information. The shape of customized bracketslots, the bracket base, and the shape of an orthodontic archwire, arecalculated in accordance with a mathematically-derived target archform.The goal of the Andreiko et al. method is to give more predictability,standardization, and certainty to orthodontics by replacing the humanelement in orthodontic appliance design with a deterministic,mathematical computation of a target archform and appliance design.Hence the '562 patent teaches away from an interactive, computer-basedsystem in which the orthodontist remains fully involved in patientdiagnosis, appliance design, and treatment planning and monitoring.

More recently, in the late 1990's Align Technologies began offeringtransparent, removable aligning devices as a new treatment modality inorthodontics. In this system, a plaster model of the dentition of thepatent is obtained by the orthodontist and shipped to a remote appliancemanufacturing center, where it is scanned with a laser. A computer modelof the dentition in a target situation is generated at the appliancemanufacturing center and made available for viewing to the orthodontistover the Internet. The orthodontist indicates changes they wish to maketo individual tooth positions. Later, another virtual model is providedover the Internet and the orthodontist reviews the revised model, andindicates any further changes. After several such iterations, the targetsituation is agreed upon. A series of removable aligning devices orshells are manufactured and delivered to the orthodontist. The shells,in theory, will move the patient's teeth to the desired or targetposition.

U.S. Pat. No. 6,632,089 to Rubbert discloses an interactive,software-based treatment planning method to correct a malocclusio. Themethod can be performed on an orthodontic workstation in a clinic or ata remote location such as a lab or precision appliance manufacturingcenter. The workstation stores a virtual three-dimensional model of thedentition of a patient and patient records. The virtual model ismanipulated by the user to define a target situation for the patient,including a target archform and individual tooth positions in thearchform. Parameters for an orthodontic appliance, such as the locationof orthodontic brackets and resulting shape of a customized orthodonticarchwire, are obtained from the simulation of tooth movement to thetarget situation and the placement position of virtual brackets.

The key to planning optimal orthodontic, other and oral treatments isobtaining three dimensional images of actual roots of teeth of apatient. Practitioners have produced three dimensional models of rootsfor treatment planning from x-rays and tooth templates; however, thereis no assurance that such three dimensional models of roots do reallyrepresent the anatomy of actual roots.

Suzanne U. McCornick and Stephanie J, Drew in an article published inJournal of Oral and Maxillofacial Surgery, “Virtual Model Surgery forEfficient Planning and Surgical Performance”, published March 2011, Vol.69, Number 3, pp. 638-644, disclose a modeling technique for creating athree dimensional computer based model of a patient for planningtreatment for a patient. Their approach requires overlaying digitaldental models obtained from a laser surface scanner over the CT/CBCTscan and align the skeletal components into natural head position usingan orientation sensor. The laser scan model is obtained by scanning astone model of the patient's teeth. Also a bite fork, with a face bowwith radiographic markers, is used to obtain the information regardingthe bite of the patient. While this approach shows some promisingpossibilities, it basically requires fusion of models produced byvarious devices in to a single composite model. The authors did notdisclose any method for producing a three dimensional model of thepatient's dentition enabling creation of three dimensional images of thepatient's tooth roots.

In orthodontic treatment planning, virtual models of the dentition of apatient play a key role and are extremely important. By-and-large so farthe models created from surface scan are used. These models lack in theareas or roots, bones and soft tissues. Therefore a need exists to forthe virtual three dimensional models of dentition including tooth rootsand surrounding anatomy which can be used in planning orthodontictreatment based upon very important information concerning threedimensional anatomy of craniofacial and dentition structures of apatient. The present invention meets this need.

SUMMARY OF THE INVENTION

Surface scans of a patient's dentition are obtained using in-vivoscanning or other types of scanning such as scanning an impression ofthe patient's dentition or scanning a physical model of the patient'sdentition. There are number of scanning devices available to accomplishthis task. Volume scans of the patient's dentition are obtained usingCone Beam Computed tomography (CBCT) or Magnetic Resonant Tomography(MRI) imaging equipment. Surface scans provide data for modelingbasically tooth crowns; whereas volume scans provide data for crowns aswell as roots, bones, soft tissues and airways. The invention disclosedherein combines the surface scan data with the volume scan data togenerate three dimensional models of a patient's dentition andsurrounding anatomy including roots, bones, soft tissues, airways, etc.Both method and workstation for generating these virtual models aredisclosed. The procedure can be summarized as follows:

-   -   a. obtain a intraoral surface scan or impression/plaster scan of        a part or the dentition within the jaw of a patient;    -   b. obtain volume scan of the same patient's dentition including        roots, bones, soft tissues, etc.

Note: both types of scans, i.e., surface and volume, have to representthe same patient in the same or similar condition.

-   -   c. generate a surface representation of the dentition from a the        volume scan. The surface representation can be generated by        thresh-holding or by any other method to generate a surface from        volume data. The scan data is processed to produce a mesh.    -   d. register the whole or a part of the surface scan into the        surface extracted from volume scan data. This scan a\data is        also processed to produce a mesh. The precondition for the        registration is that an overlap between both types of scans,        i.e., surface and volume, must exist that is sufficiently        similar in both scans.    -   e. merging both meshes together, so that the data from the        surface scan replaces the data built from the volume scan.    -   The workstation receives both types of scanning data and        provides software tools for processing each type of data as well        as for merging them together. The results are displayed on the        workstation display.        The preferred embodiment of the invention discloses an apparatus        comprising, in combination,

a computer-readable medium storing data representing a unified threedimensional virtual craniofacial and dentition model of actual, as-isstatic and functional anatomy of a patient, the data comprising:

-   -   (a) data representing facial bone structure of the patient        including the upper jaw and lower jaw;    -   (b) data representing facial soft tissue of the patient;    -   (c) data representing teeth including crowns and roots of the        patient, the data including information of the position of the        roots relative to each other and relative to the facial bone        structure of the patient including the upper jaw and the lower        jaw;    -   The data representing parts (a), (b) and (c) of unified three        dimensional virtual craniofacial and dentition model of the        patient are constructed solely from digital data obtained by        scanning as-is anatomy of craniofacial and dentition structures        of the patient with a volume scanning device;    -   (d) data representing three dimensional virtual models of the        patient's upper and lower gingiva, wherein the data represent        three dimensional virtual models of the patient's upper and        lower gingiva are constructed from scanning the patient's upper        and lower gingiva either (a) with a volume scanning device,        or (a) with a surface scanning device; the data (d) subsequently        associated with data (c); and    -   (e) data representing function of the patient's jaw movements        and smile; wherein the data representing function of the        patient's jaw movements and smile are obtained through video        imaging, jaw tracking, or photographs;    -   wherein data (a), (b), (c), (d) and (e) are represented in the        medium as individual static and/or dynamic anatomical object(s)        of the patient; and    -   a viewing program for viewing data (a), (b) (c), (d) and (e) on        a display of the workstation wherein data (a), (b) (c), (d)        and (e) can be displayed individually or in any combination on        command of a user of the workstation using the viewing program.

The preferred embodiment of the invention discloses an apparatuscomprising, in combination,

-   -   a computer-readable medium storing data representing a unified        three dimensional virtual craniofacial and dentition model of        actual, as-is static and functional anatomy of a patient, the        data comprising:

(a) data representing facial bone structure of the patient including theupper jaw and lower jaw;

(b) data representing facial soft tissue of the patient;

(c) data representing teeth including crowns and roots of the patient,the data including information of the position of the roots relative toeach other and relative to the facial bone structure of the patientincluding the upper jaw and the lower jaw;

The data representing parts (a), (b) and (c) of unified threedimensional virtual craniofacial and dentition model of the patient areconstructed solely from digital data obtained by scanning as-is anatomyof craniofacial and dentition structures of the patient with a volumescanning device;

(d) data representing three dimensional virtual models of the patient'supper and lower gingiva, wherein the data represent three dimensionalvirtual models of the patient's upper and lower gingiva are constructedfrom scanning the patient's upper and lower gingiva either (a) with avolume scanning device, or (a) with a surface scanning device; the data(d) subsequently associated with data (c); and

(e) data representing function of the patient's jaw movements and smile;wherein the data representing function of the patient's jaw movementsand smile are obtained through video imaging, jaw tracking, orphotographs;

wherein data (a), (b), (c), (d) and (e) are represented in the medium asindividual static and/or dynamic anatomical object(s) of the patient;and

a viewing program for viewing data (a), (b) (c), (d) and (e) on adisplay of the workstation wherein data (a), (b) (c), (d) and (e) can bedisplayed individually or in any combination on command of a user of theworkstation using the viewing program.

Another preferred embodiment of the invention discloses method ofplanning comprehensive treatment of a patient, having a craniofacialdeformity, skeletal abnormalities, soft tissue abnormalities, dentalmalocclusion, and dysfunction, by a practitioner, using a workstationcomprising a computing platform having a graphical user interface, aprocessor and a computer storage medium containing digitized recordspertaining to a patient, said digitized records including image data,and a set of software instructions providing graphical user interfacetools for access to said digitized records, the method comprising thesteps of:

(a) loading into the workstation a unified three dimensional virtualcraniofacial and dentition model of said patient; wherein said unifiedthree dimensional virtual craniofacial and dentition model comprises:

-   -   (i) facial bone structure including upper jaw and lower jaw;    -   (ii) facial soft tissue;    -   (iii) teeth including crowns and roots; wherein said roots are        positioned relative to each other and relative to bones of said        upper jaw and bones of said lower jaw;    -   (iv) upper and lower gingiva; and    -   (v) data representing function of the patient's jaw movements        and smile; wherein said data representing function of the        patient's jaw movements and smile are obtained through video        imaging, jaw tracking, or photographs;    -   wherein the virtual model comprising elements from (i), (ii),        (iii), (iv) and (v) are individual and separate data objects and        viewable individually or in any combination via the graphical        user interface;

(b) examining said unified three dimensional virtual craniofacial anddentition model of said patient;

(c) identifying one or more abnormalities requiring surgery forcorrecting said one or more abnormalities in said patient's craniofacialand/dentition;

(d) creating a post-surgery desired setup of said patient's teeth,including movements of one or more of said teeth and movements withinsaid bone structure, for curing said malocclusion;

(e) creating a pre-surgical setup of said patient's teeth whileretaining said movements of one or more of said teeth, but removing saidmovements within said bone structure; both from said post-surgerydesired setup;

(f) creating a pre-surgical setup of said patient's teeth whileretaining said movements of one or more of said teeth, but removing saidmovements within said bone structure;

(g) adjusting said movements of one or more of said teeth in saidpre-surgical setup thereby allowing room for said surgery for removingsaid one or more abnormalities; and creating adjusted pre-surgicalsetup;

(h) designing orthodontic appliances for said patient in accordance withsaid adjusted pre-surgical setup;

(i) designing orthodontic appliances for said patient in accordance withsaid post-surgical setup;

(j) designing surgical appliances for said patient in accordance withsaid pre-surgical setup;

(k) designing surgical appliances for said patient in accordance withsaid post-surgical setup; and

(l) sending data for manufacturing appliances.

Another preferred embodiment of the invention discloses a method oforthodontic treatment planning for a patient having tooth-rootsabnormalities, using a workstation having a processing device, a storagedevice, and an user interface with a display, comprising the steps of:

-   -   (a) obtaining a three dimensional virtual model of dentition of        the patient; wherein the virtual model of dentition is        constructed solely from volume scanned digital images of actual        craniofacial and dentition structure of the patient, and        comprises the patient's teeth with three-dimensional crowns and        three-dimensional roots and three-dimensional upper and lower        jaw bones;    -   (b) identifying the tooth-roots abnormalities; and    -   (c) planning corrective treatment steps to cure the tooth-roots        abnormalities.

BRIEF DESCRIPTION OF THE DRAWINGS

Presently preferred embodiments of the invention are described below inreference to the appended drawings, wherein like reference numeralsrefer to like elements in the various views, and in which:

FIG. 1 is block diagram of a system for creating a three-dimensionalvirtual patient model and for diagnosis and planning treatment of thepatient.

FIGS. 2-6 show data from the CBCT volume scan of a patient at differentstages of processing.

FIG. 2 shows a slice from a CBCT scan of a patient's dentition. Alsoshown in this figure is a photographic view of a tooth with a metalfilling that distorts the CBCT data.

FIG. 3 shows a volume scan from CBCT imaging of the patient's dentition.Also shown is the same tooth with a metal filling previously shown inFIG. 2.

FIG. 4 shows a volume scan from CBCT with noise caused by slightmovement of the dentition by the patient while scanning was in progress.

FIG. 5 shows a side view of both jaws of a patient with teeth, roots andjaw bones of the dentition of the patient developed from the CBCT volumescan data.

FIG. 6 shows a front view of both jaws of the patient with teeth, rootsand jaw bones of the dentition of the patient developed from the CBCTvolume scan data. The brackets placed on the patient's teeth are shownas well.

FIGS. 7-8 show data from surface scan of a patient at different stagesof processing.

FIG. 7 shows surface scan data of the partial dentition of a patient.Also shown is the same tooth with a metal filling previously shown inFIG. 2.

FIG. 8 shows final modeling of teeth obtained from surface scanning ofthe dentition of a patient. While tooth crowns are displayed in themodel, tooth roots and jaw bones are missing.

FIGS. 9-14 show data from the CBCT volume scan of a patient beingcombined with the data from surface scan of the same patient atdifferent stages of processing.

FIG. 9 shows a part of the surface scan data previously shown in FIG. 7super imposed over the volume scan data previously shown in FIG. 3. Alsoshown is the same tooth with a metal filling previously shown in FIG. 2.

FIG. 10 shows another example of a part of surface scan data superimposed over volume scan data. Also shown in this figure is aphotographic view of the mouth of the patient that was scanned.

FIG. 11 shows a finished model of the teeth with roots of a patientobtained by registering the mesh data from the surface scan with themesh data from the volume scan of the dentition of a patient.

FIGS. 12 and 13 show different views of the teeth model shown in FIG.11.

FIG. 14 shows the three-dimensional final model teeth with bracketsplaced on the teeth.

FIGS. 15-23 show various views of unified three dimensional virtualcraniofacial and dentition model derived from the volume scan data.

FIG. 15 shows front view upper and lower jaw-bones and teeth of apatient.

FIG. 16 shows right bucal view of upper and lower jaw-bones and teeth ofa patient.

FIG. 17 shows front view of upper and lower jaws with a portion of thejaw-bone removed so that more of the teeth of the patient can be seen.

FIG. 18 shows front view of the upper and lower jaws and facial bonewith modeled teeth, all obtained from the volume scan of the patient.

FIG. 19 shows left bucal view of the upper and lower jaws and facialbone with modeled teeth, all obtained from the volume scan of thepatient.

FIG. 20 shows teeth with crowns and roots in three-dimensions modeledfrom the volume scan data of the patient.

FIG. 21 shows facial soft tissue model of the patient obtained from thevolume scan data of the patient.

FIG. 22 shows a combination of facial tissue and teeth of the patient.

FIG. 23 shows ceff view of facial tissue plus jaw and facial bones andvertebra model of a patient with modeled teeth; all derived from thevolume scan data of the patient.

FIGS. 24-25 show modeling of gum tissue and its integration with the jawbones and teeth of a patient.

FIG. 24 shows front view of the three dimensional craniofacial model ofthe patient, obtained by CBCT scanning showing bones and teeth. Thefigure also shows gingiva bite model of the patient registered with thejaws and teeth of the patient. This step is necessary because a jawseparating mouth piece is inserted in the patient's mouth while scanningwith CBCT, which keeps the jaws of the patient open, and preventsscanning the bite. Therefore, the virtual model of the upper and lowergingiva of the patient, obtained from surface scanning data, as shown inFIG. 25, is registered with bones and teeth models of the patientobtained from the volume scan data.

FIG. 25 shows front view of the virtual model of the upper and lowergingiva, obtained from surface scanning data, as shown in FIG. 24integrated with bones and teeth models of the patient derived from thevolume scan data.

FIGS. 26-30 show modeling of brackets bonded to the patients teeththrough volume scanning.

FIG. 26 shows front view of the three dimensional models of the bracketsplaced on the patients teeth obtained through volume scanning of thepatient.

FIG. 27 shows another view of the layout of the bracket models presentedin FIG. 26.

FIG. 28 shows the brackets in the form of line drawings.

FIG. 29 shows models of the specific brackets, derived from the bracketimpressions, mounted on the models of the crowns of the patient.

FIG. 30 shows models of the specific brackets, derived from the bracketimpressions, mounted on the models of the teeth of the patient, alongwith the models of the jaw and other facial bones of the patient.

FIGS. 31-35 show model of the patient's bite obtained through surfacescanning, and its integration with the patient's jaws and teeth.

FIG. 31 shows model of the patient's bite obtained through surfacescanning.

FIG. 32 shows integration of the bite scan model with the jaw bones andteeth of the patient. The bite is shown in the closed position in thisfigure.

FIG. 33 shows bite in an open position along with the facial and jawbone structures of the patient in the right bucal view.

FIG. 34 also shows from the right bucal view bite in a further openposition compared to the bite in FIG. 33, along with the facial and jawbone structures of the patient. Form the volume scan data, it ispossible to identify the portion of the jaw bone which functions like ahinge for moving the jaw; and thereby simulate the movement of the jaws.

FIG. 35 shows a snap shot of the bite simulation from the left bucalview.

FIGS. 36A-36T show planning movement of one or more teeth includingcrowns and roots along with soft tissue, e.g., gingiva, and bone inorder to realize the desired objectives.

FIG. 36A shows image of patient's tooth crowns.

FIG. 36B shows patient's crowns with roots.

FIG. 36C shows patient's crowns and gingiva. Roots are hidden behind thegingiva and the bone.

FIG. 36D shows 3-D isolated image of the gingival tissue.

FIG. 36E shows image of the upper arch with teeth and roots hiddenbehind the gingival tissue and bone. Also bone is hidden by the gingivaltissue.

FIG. 36F shows appliances on patient's teeth and one tooth beingextruded. The tooth has moved 4 mm.

FIG. 36G shows simulation of gingival tissue as the tooth comes down.Note the gingival tissue follows tooth movement.

FIG. 36H shows change in architecture of gingival tissue as a result ofthe tooth movement.

FIG. 36I shows a 2-D panorex X-ray view of a patient's teeth with rootsand brackets.

FIG. 36J shows a 2-D ceph X-ray view of a patient's teeth with roots andbrackets.

FIG. 36K shows 2-D photo of a patient's tooth crown, gum tissue andbrackets.

FIG. 36L shows a 3-D view of the same teeth with roots but without gumtissue and bone.

FIG. 36M shows a 3-D top view of the same teeth with roots but withoutgum tissue and bone. Note the root of the canine is trapped between theroots of the first premolar. This is unobservable in 2-D views.

FIG. 36N shows that if the tooth movement is not planned correctlywithout considering the location of roots in 3-D space root collisionmay occur causing root resorption as shown.

FIG. 36O shows avoidance of the root collision by planning extrusion ofthe first bicuspid.

FIG. 36P shows avoidance of the root collision by proper planning oftooth and root movement in 3-D space.

FIG. 36Q shows roots out of bone on a volumetric image.

FIG. 36R shows the movement of the roots bringing them inside the boneto gain more support for the tooth; and the appliance designed toachieve it.

FIG. 36S shows planning of bone movement for surgery with appropriate2-D and 3-D images and composites.

FIG. 36T shows the upper occlusal surfaces of the teeth and their rootsbelow, three segments to plan for 3 piece maxillary surgery have beenselected. The upper right segment has been expanded and its displacementvalue shown. Any number of segments can be chosen, and the site of theostetotmy can be defined if the roots fall in to the ostetotmy sitetheir movement can be planned presurgically away from the resection siteto avoid damage. The segments can be moved in all three planes of space.

FIGS. 37A-37N show planning of tooth shape and form and restorationswith and without orthodontic treatment to achieve maximum aesthetics asefficiently as possible.

FIG. 37A shows both a 2-D view and a 3-D view of the mal-formed tooth,and the space on either side of the mal-formed tooth. This space can beclosed orthodontically or the shape or the shape and form of the toothcan be restored.

FIG. 37B shows planning to close the space on either side by restoringthe tooth on either side by 1 mm.

FIG. 37C shows that one side of the tooth has been restored to close thespace and achieve normal tooth form.

FIG. 37D shows that both sides of the tooth have been restored to closethe space and achieve normal tooth form.

FIG. 37E shows missing second bicuspid tooth.

FIG. 37F shows dialogue box to select appropriate tooth to close space.

FIG. 37G shows tooth has been automatically selected to fill space. Notethat it is over contoured.

FIG. 37H shows that the tooth form has been shaped to look moreappropriate. Note that there is a little space left. This can be closedorthodontically by moving the molar forward.

FIG. 37I shows that the patient has crowding. Efficient resolution ofthe crowding is planned by optimizing tooth movement and restorationwith veneers.

FIG. 37J shows that the teeth have been moved orthodontically to helpresolve some of the crowding.

FIG. 37K shows that all teeth have been moved orthodontically to helpresolve some of the crowding.

FIG. 37L shows the selection of automatic veneer build up to resolve thecrowding completely and to provide better shape and form. 3-D simulationplanning has allowed the optimal planning of care from both restorativeand orthodontic perspectives with minimal tooth movement and destructionof tooth.

FIG. 37M shows fractured incisal edge. Choice of either restoring thefracture by using a template tooth, or a similar non-fractured tooth, orchoosing either the mesial or distal side of any normal shaped tooth torestore edges or line angles.

FIG. 37N shows fractured incisal edge restored in normal shape and form.

FIG. 38A shows composite image of craniofacial complex by combiningcraniofacial bones, tooth crown and roots from CBCT, gingival tissuefrom surface scanning, and facial soft tissue from 2-D photographs andtooth with roots; and the lower jaw and its teeth registered to theupper jaw and teeth using an intraoral bite registration scan.

FIG. 38B shows composite image of craniofacial complex by combiningcraniofacial bones, tooth crown and roots from CBCT, gingival tissuefrom surface scanning, and facial soft tissue from 2-D photographs andtooth with roots; and the lower jaw and its teeth registered to theupper jaw and teeth using an intraoral bite registration scan. Thefigure also shows functional movements such as mouth opening.

FIG. 38C shows volumetric 3-D image of teeth and gingival tissue inrelation to the patient's face in order to assess and plan foraesthetics.

FIG. 39A shows the dental complex comprising the roots and the crowns,which can be oriented to user defined reference planes. For examplereference from the frontal perspective has been defined by a linebetween the pupils of the eyes and perpendicular to this line.Similarly, user defined reference plane. A second reference plane hasbeen chosen from the lateral perspective. The combination of frontalreference plane and lateral reference plane, with a common coordinatesystem with respect to the model and the face allows for correctorientation in 3-D space. The face and its accompanying structures canalso be oriented by the user from any perspective including frontal andlateral.

FIG. 39B shows a side view of the dental complex showing bones, teethand a reference plane.

FIG. 39C shows different orientations of the face of the patient.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENT

Before describing the features of this invention in detail, an overviewof a unified workstation will be set forth initially. The workstationprovides software features that create two dimensional and/orthree-dimensional virtual patient model on a computer, which can be usedfor purposes of communication, diagnosis, treatment planning and designof customized appliances in accordance with a presently preferredembodiment.

The essence of the invention disclosed herein is the ability to captureimages from various sources that provide volumetric images, surfaceimages that are 3-D or 2-D in nature, and may be static or dynamic, suchas from CBCT, CAT, MRI, fMRI, ultrasound device, cameras that providestill photos, white light and laser based surface scanners, videocameras providing video images, tracking devices and digital camera.Images from these sources are combined as needed to create a unifiedsimulation model of the craniofacial and dental facial complex, tofacilitate diagnosis, communication, planning treatment and design ofappliances for treating craniofacial and dento facial problems. Withthese images a composite structure of the face can be constructed withdynamic or static behavioral properties. One can also track function orjaw movement and simulate the functional movements, e.g., smile movementof the lower jaw etc.

The global positioning of the entire face with respect to itssurroundings can be set by the user for planning purposes. In addition,the relative position of each of the structural elements, such as theupper jaw and its teeth when captured independently can be oriented withrespect to any other structure such as the soft tissue face by usingspecific anatomical land marks or user defined reference planes, eitherin 2-D or 3-D space. Furthermore, the relationship of the lower jaw andits accompanying teeth can be registered with respect to the upper jawusing a combination of registration techniques. For instance, the biteregistration can be recorded by taking an intraoral surface scan of theteeth together and using it as a template to register the relationshipof the upper jaw and the lower jaw from a CBCT volumetric scan.

Most importantly from volumetric data, one can extract three dimensionalstructural data which may include crowns and roots of teeth, bone, softtissue, e.g., gingiva and facial soft tissue and appliances attached toany of these structures, such as orthodontic brackets, implants, etc.Each of these structural elements can be independently manipulated inthree dimensional space to construct a treatment plan, and design theappropriate device for correction of a problem. Furthermore, theinterdependencies of the treatment between these various structuralcomponents can be modeled to design a holistic treatment plan. Specificrelationships between the various structural components can be definedby choosing an appropriate reference plane and capturing the spatialrelationships between specific structures. The treatment design mayinclude repositioning, restoring, replacing of any of the structuralelements in 2-D or 3-D space. Also function can be simulated or modeledbased upon captured data to achieve the desired goals, e.g., the teethwith their roots can be appropriately positioned in the bone towithstand the stresses of jaw movement or the position of the jaw jointi.e. the condyle is in harmony with the position of the teeth to preventany source of dysfunction or breakdown of the structural elements.Mechanical analysis, such as finite element method, may also be used tobetter understand the nature of stresses and strains imposed on thestructural elements to design better treatment. All changes may bemeasured with respect to defined planes of reference to providenumerical output to design a variety of customized treatment devices,such as orthodontic brackets, orthodontic archwires, surgical bitesplints, surgical fixation plates, implants, condylar prosthesis, bonescrews, periodontal stents, mouth guards, bite plates, removableorthodontic appliances, crowns, bridges, dentures, partial dentures,obturataors, temporary anchorage devices from either natural orsynthetic substances using printing devices, such as SLA or milling orrobotic manufacturing. Any type of dental, orthodontic, restorative,prosthodontic or surgical device which may be tissue borne, dentalborne, osseous borne, can be designed in combination, or singularly inserial or in parallel, e.g., indirect bonding trays that allow bondingof brackets, and are also designed to guide implant insertion.Furthermore, if the patient requires surgery, splints, fixation plates,boney screws may all be designed and manufactured simultaneously. Thenumerical output of the treatment plan can be used to drive navigationalsystems for performing any procedure. Simulations can be used to trainand build skills or examine proficiency. The numerical output of thetreatment design can be used to drive robots to perform surgicalprocedures. Furthermore this output can be used to create a solid modelrepresentation of the treatment plan using printing or millingtechniques.

Template data or normative data stored in memory can be used to plan anyof the structural changes or design of the devices. In addition,reference data from the non-affected structural elements may be used astemplates to provide design parameters to plan and correct the affectedside.

One can also replace or remove any of the structures to achieve thedesired goal, e.g., extraction of teeth, root amputation, sinus lift,veneers, inter-proximal reduction, etc. The codependency of movement ofone object and its effect on another can also be simulated for all threetissue types eg when the tooth moves how does it affect the gum softtissue when the tooth moves where does the root move in reference to thebone or how does the bone change how does the face change when the bonesmove. All types of planning can be executed by different modalities orprofessionals in an interactive manner asynchronously or synchronously.

In summary, the invention disclosed herein provides the ability to plancrowns with roots thereby optimizing the planning by changing the rootposition so that the crown planned is designed such that axial forcesare transmitted to the roots to add to the stability of the crownminimizing aberrant forces that can lead to root fracture, crownfracture, and breakdown of the periodontium or bone. Similarly forsurgical patients one can plan root positions so that the surgeon cancut between the roots and prevent damage besides planning the movementof the bones. Similarly for implants one can move the roots in adesirable location so that the implant when inserted doesn't damage theroots. The user can also size the implants correctly so that they don'tencroach on root space. All this planning would be impossible if theroots were not made separate objects that could move. Finally one canmove the roots preferentially to create bone. As one extrudes a root onecan create bone. Similarly one can change the gum tissue architecture bymoving roots and for orthodontic movement one can avoid moving rootswhere there is no bone or selectively move teeth to prevent rootcollision or move roots away from areas where there is lack of bone intowhere there is as one plans to move them towards their finaldestination. Again not only can one plan tooth movement but bonemovement and soft tissue gum and face as well to achieve the goals. Onecan alter the spatial position of all the objects which are extracted,change their shape form and volume to restore and or reconstruct. Onecan sculpt or remove selectively any region gum soft issue bonedentition. Although one can use a fusion technique, the preferredembodiment is to extract the data from the CBCT for bone and dentitionwith roots at a minimum. One can take partial intramural scans wheredistortion is expected, e.g., large metal crowns or fillings or one canscan an impression in those areas or plaster limited to the region ofinterest.

The images of the roots can be taken with CBCT and affixed to crownstaken by scanning intramurally impressions or models. The preferredprocess does not require fusing a model of the dentition into the crankfacial structure. All needed information can be captured in one shot andextract individual features. The invention disclosed herein captures thedental and osseous and soft tissue as one and segregate them in toindividual components for planning treatment. The optimization of thetreatment plan can be achieved by using different approaches, e.g.,correcting crowding by minimizing tooth movement and planning veneers orminimizing tooth preparation for veneer construction by positioning theteeth appropriately. This can be said for any structure and the decisioncan be driven by the patients need, time constraints, cost risk benefit,skill of operator, etc.

Many of the details and computer user interface tools which apractitioner may use in adjusting tooth position, designing applianceshape and location, managing space between teeth, and arriving at afinish tooth position using interaction with a computer storing anddisplaying a virtual model of teeth are set forth in the priorapplication Ser. No. 09/834,412 filed Apr. 13, 2001, and in publishedOraMetrix patent application WO 01/80761, the contents of which areincorporated by reference herein. Other suites of tools and functionsare possible and within the scope of the invention. Such details willtherefore be omitted from the present discussion.

General Description

A unified workstation environment and computer system for diagnosis,treatment planning and delivery of therapeutics, especially adapted fortreatment of craniofacial structures, is described below. In onepossible example, the system is particularly useful in diagnosis andplanning treatment of an orthodontic patient. Persons skilled in the artwill understand that the invention, in its broader aspects, isapplicable to other craniofacial disorders or conditions requiringsurgery, prosthodontic treatment, restorative treatment, etc.

A presently preferred embodiment is depicted in FIG. 1. The overallsystem 100 includes a general-purpose computer system 10 having aprocessor (CPU 12) and a user interface 14, including screen display 16,mouse 18 and keyboard 20. The system is useful for planning treatmentfor a patient 34.

The system 100 includes a computer storage medium or memory 22accessible to the general-purpose computer system 10. The memory 22,such as a hard disk memory or attached peripheral devices, stores two ormore sets of digital data representing patient craniofacial imageinformation. These sets include at least a first set of digital data 24representing patient craniofacial image information obtained from afirst imaging device and a second set of digital data 26 representingpatient craniofacial image information obtained from a second imagedevice different from the first image device. The first and second setsof data represent, at least in part, common craniofacial anatomicalstructures of the patient. At least one of the first and second sets ofdigital data normally would include data representing the externalvisual appearance or surface configuration of the face of the patient.

In a representative and non-limiting example of the data sets, the firstdata set 24 could be a set of two dimensional color photographs of theface and head of the patient obtained via a color digital camera 28, andthe second data set is three-dimensional image information of thepatient's teeth, acquired via a suitable scanner 30, such as a hand-heldoptical 3D scanner, or other type of scanner. The memory 22 may alsostore other sets 27 of digital image data, including digitized X-rays,MRI or ultrasound images, CT scanner, CBCT scanner, jaw tracking device,scanning device, video camera, etc., from other imaging devices 36. Theother imaging devices need not be located at the location or site of theworkstation system 100. Rather, the imaging of the patient 34 with oneor other imaging devices 36 could be performed in a remotely locatedclinic or hospital, in which case the image data is obtained by theworkstation 100 over the Internet 37 or some other communicationsmedium, and stored in the memory 22.

The system 100 further includes a set of computer instructions stored ona machine-readable storage medium. The instructions may be stored in thememory 22 accessible to the general-purpose computer system 10. Themachine-readable medium storing the instructions may alternatively be ahard disk memory 32 for the computer system 10, external memory devices,or may be resident on a file server on a network connected to thecomputer system, the details of which are not important. The set ofinstructions, described in more detail below, comprise instructions forcausing the general computer system 10 to perform several functionsrelated to the generation and use of the virtual patient model indiagnostics, therapeutics and treatment planning.

These functions include a function of automatically, and/or with the aidof operator interaction via the user interface 14, superimposing thefirst set 24 of digital data and the second set 26 of digital data so asto provide a composite, combined digital representation of thecraniofacial anatomical structures in a common coordinate system. Thiscomposite, combined digital representation is referred to hereinoccasionally as the “virtual patient model,” shown on the display 16 ofFIG. 1 as a digital model of the patient 34. Preferably, one of the sets24, 26 of data includes photographic image data of the patient's face,teeth and head, obtained with the color digital camera 28. The other setof data could be intra-oral 3D scan data obtained from the hand-heldscanner 30, CT scan data, X-Ray data, MRI, etc. In the example of FIG.1, the hand-held scanner 30 acquires a series of images containing 3Dinformation and this information is used to generate a 3D model in thescanning node 31, in accordance with the teachings of the published PCTapplication of OraMetrix, PCT publication no. WO 01/80761, the contentof which is incorporated by reference herein. Additional data sets arepossible, and may be preferred in most embodiments. For example thevirtual patient model could be created by a superposition of thefollowing data sets: intra-oral scan of the patient's teeth, gums, andassociated tissues, X-Ray, CT scan, intra-oral color photographs of theteeth to add true color (texture) to the 3D teeth models, and colorphotographs of the face, that are combined in the computer to form a 3Dmorphable face model. These data sets are superimposed with each other,with appropriate scaling as necessary to place them in registry witheach other and at the same scale. The resulting representation can bestored as a 3D point cloud representing not only the surface on thepatient but also interior structures, such as tooth roots, bone, andother structures. In one possible embodiment, the hand-held in-vivoscanning device is used which also incorporates a color CCD camera tocapture either individual images or video.

The software instructions further includes a set of functions orroutines that cause the user interface 16 to display the composite,combined digital three-dimensional representation of craniofacialanatomical structures to a user of the system. In a representativeembodiment, computer-aided design (CAD)-type software tools are used todisplay the model to the user and provide the user with tools forviewing and studying the model. Preferably, the model is cable of beingviewed in any orientation. Tools are provided for showing slices orsections through the model at arbitrary, user defined planes.Alternatively, the composite digital representation may be printed outon a printer or otherwise provided to the user in a visual form.

The software instructions further include instructions that, whenexecuted, provide the user with tools on the user interface 14 forvisually studying, on the user interface, the interaction of thecraniofacial anatomical structures and their relationship to theexternal, visual appearance of the patient. For example, the toolsinclude tools for simulating changes in the anatomical position or shapeof the craniofacial anatomical structures, e.g., teeth, jaw, bone orsoft tissue structure, and their effect on the external, visualappearance of the patient. The preferred aspects of the software toolsinclude tools for manipulating various parameters such as the age of thepatient; the position, orientation, color and texture of the teeth;reflectivity and ambient conditions of light and its effect on visualappearance. The elements of the craniofacial and dental complex can beanalyzed quickly in either static format (i.e., no movement of theanatomical structures relative to each other) or in an dynamic format(i.e., during movement of anatomical structures relative to each other,such as chewing, occlusion, growth, etc.). To facilitate such modelingand simulations, teeth may be modeled as independent, individuallymoveable 3 dimensional virtual objects, using the techniques describedin the above-referenced OraMetrix published PCT application, WO01/80761.

The workstation environment provided by this invention provides apowerful system and for purposes of diagnosis, treatment planning anddelivery of therapeutics. For example, the effect of jaw and skullmovement on the patient's face and smile can be studied. Similarly, themodel can be manipulated to arrive at the patient's desired feature andsmile. From this model, and more particularly, from the location andposition of individual anatomical structures (e.g., individual toothpositions and orientation, shape of arch and position of upper and lowerarches relative to each other), it is possible to automatically backsolve for or derive the jaw, tooth, bone and/or soft tissue correctionsthat must be applied to the patient's initial, pre-treatment position toprovide the desired result. This leads directly to a patient treatmentplan.

These simulation tools, in a preferred embodiment, compriseuser-friendly and intuitive icons 35 that are activated by a mouse orkeyboard on the user interface of the computer system 10. When theseicons are activated, the software instruction provide pop-up, menu, orother types screens that enable a user to navigate through particulartasks to highlight and select individual anatomical features, changetheir positions relative to other structures, and simulate movement ofthe jaws (chewing or occlusion). Examples of the types of navigationaltools, icons and treatment planning tools for a computer user interfacethat may be useful in this process and provide a point of departure forfurther types of displays useful in this invention are described in thepatent application of Rudger Rubbert et al., Ser. No. 09/835,039 filedApr. 13, 2001, the contents of which are incorporated by referenceherein.

The virtual patient model, or some portion thereof, such as datadescribing a three-dimensional model of the teeth in initial and targetor treatment positions, is useful information for generating customizedorthodontic appliances for treatment of the patient. The position of theteeth in the initial and desired positions can be used to generate a setof customized brackets, and customized flat planar archwire, andcustomized bracket placement jigs, as described in the above-referencedAndreiko et al. patents. Alternatively, the initial and final toothpositions can be used to derive data sets representing intermediatetooth positions, which are used to fabricate transparent aligning shellsfor moving teeth to the final position, as described in theabove-referenced Chisti et al. patents. The data can also be used toplace brackets and design a customized archwire as described in thepreviously cited application Ser. No. 09/835,039.

To facilitate sharing of the virtual patient model among specialists anddevice manufacturers, the system 100 includes software routines andappropriate hardware devices for transmitting the virtual patient modelor some subset thereof over a computer network. The system's softwareinstructions are preferably integrated with a patient management programhaving a scheduling feature for scheduling appointments for the patient.The patient management program provides a flexible scheduling of patientappointments based on progress of treatment of the craniofacialanatomical structures. The progress of treatment can be quantified. Theprogress of treatment can be monitored by periodically obtaining updatedthree-dimensional information regarding the progress of treatment of thecraniofacial features of the patient, such as by obtaining updated scansof the patient and comparison of the resulting 3D model with theoriginal 3D model of the patient prior to initiation of treatment.

Thus, it is contemplated that system described herein provides a set oftools and data acquisition and processing subsystems that togetherprovides a flexible, open platform or portal to a variety of possibletherapies and treatment modalities, depending on the preference of thepatient and the practitioner. For example, a practitioner viewing themodel and using the treatment planning tools may determine that apatient may benefit from a combination of customized orthodonticbrackets and wires and removable aligning devices. Data from the virtualpatient models is provided to diverse manufacturers for coordinatedpreparation of customized appliances. Moreover, the virtual patientmodel and powerful tools described herein provide a means by which thecomplete picture of the patient can be shared with other specialists(e.g., dentists, maxilla-facial or oral surgeons, cosmetic surgeons,other orthodontists) greatly enhancing the ability of diversespecialists to coordinate and apply a diverse range of treatments toachieve a desired outcome for the patient. In particular, the overlay orsuperposition of a variety of image information, including 2D X-Ray, 3Dteeth image data, photographic data, CT scan data, and other data, andthe ability to toggle back and forth between these views and simulatechanges in position or shape of craniofacial structures, and the abilityto share this virtual patient model across existing computer networks toother specialists and device manufacturers, allows the entire treatmentof the patient to be simulated and modeled in a computer. Furthermore,the expected results can be displayed beforehand to the patient andchanges made depending on the patient input.

With the above general description in mind, additional details ofpresently preferred components and aspects of the inventive system andthe software modules providing the functions referenced above will bedescribed next.

Integrated 3-D Modeling of Patient's Dentition from Surface Scanning andCBCT Imaging

The invention disclosed herein enables orthodontists to accuratelymeasure complex three dimensional anatomy during diagnosis and treatmentplanning of orthodontic patients. The invention enables orthodontists orusers to capture 3D scans with intraoral scanners as well as cone beamcomputed tomography (CBCT) that can capture highly precise digitalscans. The resulting digital images are downloaded to a computer, andcombined in order to create comprehensive 3-D models of the patient'sdentition, roots, bones and soft tissues thereby creating 3-D digitalteeth model and surrounding anatomy of pre-treatment mouth. Theinvention provides substantial improvement over the traditional twodimensional imaging modalities such as x-rays, photographs,cephalometric tracing for diagnosis and treatment planning.

In a preferred embodiment of the invention, scanning is done in-vivousing a white light scanner, and is non-invasive. Scanner is referenceindependent, so the object being scanned can move while being scannedand the scanned data will still be useful. Scanning can be performedagain to get the modeling information that might be missing. In order toperform bite registration, it does not require wax bite. This type ofscanning does not allow reconstruction of data. Pano and Ceph should betaken separately and imported into the image management software.

When needed, a partial scan can be taken. The scanning captures only thecoronal portion of the tooth in 3-D. However, gingiva is viewable withthis type of scanning. Scanning can be performed with orthodonticbrackets, made of either plastic or metal or a combination, placed onthe patient's teeth; as well as with one or more teeth crowns havingmetal fillings. Tooth separators are not required in order to performscanning. Excessive voids in scan data can affect tooth modeling. Thistype of scanning can be used for creating 3D models of teeth from rawscan data for diagnostic, therapeutic and final outcome evaluation.

In contrast, CBCT imaging is invasive, and requires tooth separators. Inorder to perform bite registration, a wax bite or bite block isrequired. Multiple slices taken by CBCT can be reconstructed to looklike 3-D images of teeth, jaws and even soft tissue; and Pano and Cephcan also be reconstructed from the data captured by CBCT. Partial scanis not possible with CBCT. CBCT images capture crown, root, surroundingbone and soft tissues which can be put together in three dimensions.Although gingiva can be viewed with CBCT images, the root anatomyobtained is preferred. There are several limitations while using CBCTfor imaging the patient's dentition, such as for example, (a) thebrackets are limited generally to plastic brackets since metal bracketscause image distortion, (b) metal crown fillings by-and-large cannot behandled since fillings larger than 4 mm creates noise causing imagedistortion and (c) a patient cannot move during the CBCT imagingprocedure since any motion during imaging causes blurring of imagemaking it unusable. Additionally, wax bite or bite blocks used asseparators to prevent opposing teeth from coming in contact during theCBCT imaging procedure. Excessive ‘noise’ caused by large metal objectsin patient's mouth causes distortion of images adversely affecting toothmodeling.

CBCT imaging can also be used for creating 3D models of teeth from rawimage data for diagnostic, therapeutic and final outcome evaluation.Furthermore, X-rays can be reconstructed from CBCT images.

Surface scanning as well as volume scanning by CBCT or MRI imaging eachhas some short-comings. However, they can be used in a complimentarymanner to produce 3-D digital models of patient's dentition includingteeth with roots, bones and soft tissues such as gingival, and withexcellent quality.

FIGS. 2-6 show data from the CBCT volume scan of a patient at differentstages of processing.

FIG. 2 shows a slice 302 from a CBCT scan of a patient's dentition. Alsoshown in this figure is a photographic view of a tooth 204′ with a metalfilling that distorts the CBCT data. CBCT image 204 of tooth 204′ isdamaged in slice 302 due to the metal filling. Other teeth, for exampleimage 206 of a tooth without metal filling comes out undamaged.

FIG. 3 shows a volume scan 210 from CBCT imaging of the patient'sdentition. Also shown is the same tooth 204′ with a metal fillingpreviously shown in FIG. 2. The image 204″ of the tooth 204′ with themetal filling comes out damaged; whereas, image 206″ of a tooth withoutmetal filling is acceptable.

FIG. 4 shows a volume scan 212 from CBCT imaging with noise 214 causedby slight movement of the dentition by the patient while imaging was inprogress.

Although other types of deficiencies are not shown by way of figures,one skilled in art would appreciate that volume scanning with CBCT orMill has certain inherent disadvantages.

FIG. 5 shows a side view 220 of both jaws of a patient with teeth withroots 222 and jaw bones 224 of the dentition of the patient developedfrom the CBCT volume scan data.

FIG. 6 shows a front view 230 of both jaws of the patient, lower jaw 234and upper jaw 240, with teeth and roots 236 in the lower jaw and 242 inthe upper jaw, and jaw bones 238 of the lower jaw and 244 of the upperjaw of the dentition of the patient developed from the CBCT volume scandata. The brackets placed on the patient's teeth are shown as well.

FIGS. 7-6 show data from surface scan of a patient at different stagesof processing.

FIG. 7 shows surface scan 302 of the partial dentition of a patient.Also shown is the same tooth 204′ with a metal filling previously shownin FIG. 2. Surface scan 204′″ corresponding to tooth 204′ is perfectlyacceptable even thought the tooth has metal fillings.

FIG. 8 shows final modeling of teeth, in front view 320 and side views324 and 328, obtained from surface scanning of the dentition of apatient. While tooth crowns are displayed in the model, tooth roots andjaw bones are missing.

FIGS. 9-14 show data from the CBCT volume scan of a patient beingcombined with the data from surface scan of the same patient atdifferent stages of processing.

FIG. 9 shows a part of the surface scan data 302 previously shown inFIG. 7 super imposed over the volume scan data 210 previously shown inFIG. 3. Also shown is the same tooth 204′ with a metal fillingpreviously shown in FIG. 2. Because of the surface scan data, toothrepresentation 204″ of tooth 204′ is acceptable in this case.

FIG. 10 shows another example 354 of a part of surface scan data superimposed over volume scan data 352. Also shown in this figure is aphotographic view of the mouth 350 of the patient that was scanned.

FIG. 11 shows a finished model 402 of the teeth with roots of a patientobtained by registering the mesh data from the surface scan with themesh data from the volume scan of the dentition of a patient.

FIGS. 12 and 13 show different views 410, 412, 414, 416 and 418 of theteeth model shown in FIG. 11.

FIG. 14 shows the three-dimensional final model 450 teeth with crowns460 and roots 470 having brackets 480 placed on the teeth.

FIGS. 15-23 show various views of unified three dimensional virtualcraniofacial and dentition model derived from the volume scan data.

The preferred embodiment of the invention discloses an apparatuscomprising, in combination,

-   -   a computer-readable medium storing data representing a unified        three dimensional virtual craniofacial and dentition model of        actual, as-is static and functional anatomy of a patient, the        data comprising:

(a) data representing facial bone structure of the patient including theupper jaw and lower jaw;

(b) data representing facial soft tissue of the patient;

(c) data representing teeth including crowns and roots of the patient,the data including information of the position of the roots relative toeach other and relative to the facial bone structure of the patientincluding the upper jaw and the lower jaw;

The data representing parts (a), (b) and (c) of unified threedimensional virtual craniofacial and dentition model of the patient areconstructed solely from digital data obtained by scanning as-is anatomyof craniofacial and dentition structures of the patient with a volumescanning device;

(d) data representing three dimensional virtual models of the patient'supper and lower gingiva, wherein the data represent three dimensionalvirtual models of the patient's upper and lower gingiva are constructedfrom scanning the patient's upper and lower gingiva either (a) with avolume scanning device, or (a) with a surface scanning device; the data(d) subsequently associated with data (c); and

(e) data representing function of the patient's jaw movements and smile;wherein the data representing function of the patient's jaw movementsand smile are obtained through video imaging, jaw tracking, orphotographs;

wherein data (a), (b), (c), (d) and (e) are represented in the medium asindividual static and/or dynamic anatomical object(s) of the patient;and

a viewing program for viewing data (a), (b) (c), (d) and (e) on adisplay of the workstation wherein data (a), (b) (c), (d) and (e) can bedisplayed individually or in any combination on command of a user of theworkstation using the viewing program.

FIG. 15 shows front view of upper jaw bone 500 and lower jaw bone 502and teeth 504 of a patient.

FIG. 16 shows right bucal view of upper jaw bone 510 and lower jaw bone512 and teeth 514 of a patient.

FIG. 17 shows front view of upper jaw and lower jaw with a portion ofthe jaw-bone removed so that more of the teeth 516 of the patient can beseen.

FIG. 18 shows front view of the upper jaw 520 and lower jaw 522 andfacial bone 524 with modeled teeth 526, all obtained from the volumescan of the patient.

FIG. 19 shows left bucal view of the upper jaw 530 and lower jaw 532 andfacial bone 534 with modeled teeth 536, all obtained from the volumescan of the patient.

FIG. 20 shows teeth with crowns 540 and roots 542 in three-dimensionsmodeled from the volume scan data of the patient.

FIG. 21 shows facial soft tissue model 550 of the patient obtained fromthe volume scan data of the patient.

FIG. 22 shows a combination of facial tissue 552 and teeth 554 of thepatient.

FIG. 23 shows ceff view of facial tissue 556 plus jaw bone 558 andfacial bone 560 and of a patient with modeled teeth 562; all derivedfrom the volume scan data of the patient.

FIGS. 24-25 show modeling of gum tissue and its integration with the jawbones and teeth of a patient.

FIG. 24 shows front view of the three dimensional craniofacial model ofthe patient, obtained by CBCT scanning showing bones 564 and teeth 566.The figure also shows gingiva bite model 568 and 569 of the patientregistered with the jaws and teeth of the patient. This step isnecessary because a jaw separating mouth piece is inserted in thepatient's mouth while scanning with CBCT, which keeps the jaws of thepatient open, and prevents scanning the bite. Therefore, the virtualmodel of the upper 568 and lower 569 gingiva of the patient, obtainedfrom surface scanning data, as shown in FIG. 25, is registered withbones and teeth models of the patient obtained from the volume scandata.

FIG. 25 shows the virtual model of the upper gingiva 568 and lowergingiva 569 of the patient obtained by surface scanning of the patient'sbite.

FIGS. 26-30 show modeling of brackets bonded to the patients teeththrough volume scanning.

FIG. 26 shows front view of the three dimensional models of the brackets600 placed on the patients teeth 602 obtained through volume scanning ofthe patient.

FIG. 27 shows another view of the layout of the bracket models 601 and603 presented in FIG. 26.

FIG. 28 shows the brackets in the form of line drawings 604.

FIG. 29 shows models of the specific brackets, derived from the imagesof the scanned brackets, mounted on the models of the crowns 605 of thepatient.

FIG. 30 shows models of the specific brackets 610, derived from thescanned images of the bracket, mounted on the models of the teeth 612 ofthe patient, along with the models of the jaw 614 and other facial bones616 of the patient.

FIGS. 31-35 show model of the patient's bite obtained through surfacescanning, and its integration with the patient's jaws and teeth.

Another preferred embodiment of the invention discloses method ofplanning comprehensive treatment of a patient, having a craniofacialdeformity, skeletal abnormalities, soft tissue abnormalities, dentalmalocclusion, and dysfunction, by a practitioner, using a workstationcomprising a computing platform having a graphical user interface, aprocessor and a computer storage medium containing digitized recordspertaining to a patient, said digitized records including image data,and a set of software instructions providing graphical user interfacetools for access to said digitized records, the method comprising thesteps of:

-   -   (a) loading into the workstation a unified three dimensional        virtual craniofacial and dentition model of said patient;        wherein said unified three dimensional virtual craniofacial and        dentition model comprises:

(i) facial bone structure including upper jaw and lower jaw;

(ii) facial soft tissue;

(iii) teeth including crowns and roots; wherein said roots arepositioned relative to each other and relative to bones of said upperjaw and bones of said lower jaw;

(iv) upper and lower gingiva; and

(v) data representing function of the patient's jaw movements and smile;wherein said data representing function of the patient's jaw movementsand smile are obtained through video imaging, jaw tracking, orphotographs;

wherein the virtual model comprising elements from (i), (ii), (iii),(iv) and (v) are individual and separate data objects and viewableindividually or in any combination via the graphical user interface;

(b) examining said unified three dimensional virtual craniofacial anddentition model of said patient;

(c) identifying one or more abnormalities requiring surgery forcorrecting said one or more abnormalities in said patient's craniofacialand/dentition;

(d) creating a post-surgery desired setup of said patient's teeth,including movements of one or more of said teeth and movements withinsaid bone structure, for curing said malocclusion;

(e) creating a pre-surgical setup of said patient's teeth whileretaining said movements of one or more of said teeth, but removing saidmovements within said bone structure; both from said post-surgerydesired setup;

(f) creating a pre-surgical setup of said patient's teeth whileretaining said movements of one or more of said teeth, but removing saidmovements within said bone structure;

(g) adjusting said movements of one or more of said teeth in saidpre-surgical setup thereby allowing room for said surgery for removingsaid one or more abnormalities; and creating adjusted pre-surgicalsetup;

(h) designing orthodontic appliances for said patient in accordance withsaid adjusted pre-surgical setup;

(i) designing orthodontic appliances for said patient in accordance withsaid post-surgical setup;

(j) designing surgical appliances for said patient in accordance withsaid pre-surgical setup;

(k) designing surgical appliances for said patient in accordance withsaid post-surgical setup; and

(l) sending data for manufacturing appliances.

FIG. 31 shows model of the patient's bite 620 obtained through surfacescanning.

FIG. 32 shows integration of the bite scan model 622 with the jaw bonesand teeth of the patient. The bite is shown in the closed position inthis figure.

FIG. 33 shows bite in an open position 624 along with the facial and jawbone structures of the patient in the right bucal view.

FIG. 34 also shows from the right bucal view bite 626 in a further openposition compared to the bite in FIG. 33, along with the facial and jawbone structures of the patient. Form the volume scan dat, it is possibleto identify the portion of the jaw bone which functions like a hinge formoving the jaw; and thereby simulate the movement of the jaws.

FIG. 35 shows a snap shot of the bite simulation 628 from the left bucalview.

FIGS. 36A-36T show planning movement of of one or more teeth includingcrowns and roots along with soft tissue, e.g., gingiva, and bone inorder to realize the desired objectives. Another preferred embodiment ofthe invention discloses a method of orthodontic treatment planning for apatient having tooth-roots abnormalities, using a workstation having aprocessing device, a storage device, and an user interface with adisplay, comprising the steps of:

(a) obtaining a three dimensional virtual model of dentition of thepatient; wherein the virtual model of dentition is constructed solelyfrom volume scanned digital images of actual craniofacial and dentitionstructure of the patient, and comprises the patient's teeth withthree-dimensional crowns and three-dimensional roots andthree-dimensional upper and lower jaw bones;

(b) identifying the tooth-roots abnormalities; and

(c) planning corrective treatment steps to cure the tooth-rootsabnormalities.

FIG. 36A shows image of patient's tooth crowns 630.

FIG. 36B shows patient's crowns with roots 632.

FIG. 36C shows patient's crowns 634 and gingiva 636. Roots are hiddenbehind the gingiva and the bone.

FIG. 36D shows 3-D isolated image 637 of the gingival tissue.

FIG. 36E shows image of the upper arch 638 with teeth and roots hiddenbehind the gingival tissue and bone. Also bone is hidden by the gingivaltissue.

FIG. 36F shows appliances on patient's teeth and one tooth 640 beingextruded. The tooth 640 has moved 4 mm.

FIG. 36G shows simulation of gingival tissue 642 as the tooth 644 comesdown. Note the gingival tissue follows tooth movement.

FIG. 36H shows change in architecture of gingival tissue 646 as a resultof the tooth movement.

FIG. 36I shows a 2-D panorex X-ray view 639 of a patient's teeth withroots and brackets.

FIG. 36J shows a 2-D ceph X-ray view 640 of a patient's teeth with rootsand brackets.

FIG. 36K shows 2-D photo 641 of a patient's tooth crown, gum tissue andbrackets.

FIG. 36L shows a 3-D view 642 of the same teeth with roots but withoutgum tissue and bone.

FIG. 36M shows a 3-D top view of the same teeth with roots but withoutgum tissue and bone. Note the root of the canine 700 is trapped betweenthe roots of the first premolar 702. This is unobservable in 2-D views.

FIG. 36N shows that if the tooth movement is not planned correctlywithout considering the location of roots in 3-D space root collision710 may occur causing root resorption as shown.

FIG. 36O shows avoidance of the root collision by planning extrusion ofthe first bicuspid 720.

FIG. 36P shows avoidance of the root collision 724 by proper planning oftooth and root movement in 3-D space.

FIG. 36Q shows roots 730 out of bone on a volumetric image.

FIG. 36R shows the movement of the roots 740 bringing them inside thebone to gain more support for the tooth; and the appliance 742 designedto achieve it.

FIG. 36S shows planning of bone movement for surgery with appropriate2-D and 3-D images 744, 745 and composites 746, 747.

FIG. 36T shows the upper occlusal surfaces of the teeth and their rootsbelow, three segments 750, 754 and 758 to plan for 3 piece maxillarysurgery have been selected. The upper right segment has been expandedand its displacement value shown. Any number of segments can be chosen,and the site of the ostetotmy can be defined if the roots fall in to theostetotmy site their movement can be planned pre-surgically away fromthe resection site to avoid damage. The segments can be moved in allthree planes of space.

FIGS. 37A-37N show planning of tooth shape and form and restorationswith and without orthodontic treatment to achieve maximum esthatics asefficiently as possible.

FIG. 37A shows both a 2-D view 760 and a 3-D view 762 of the mal-formedtooth, and the space on either side of the mal-formed tooth. This spacecan be closed orthodontically or the shape or the shape and form of thetooth can be restored.

FIG. 37B shows planning to close the space on either side by restoringthe tooth 770 on either side by 1 mm.

FIG. 37C shows that one side 774 of the tooth has been restored to closethe space and achieve normal tooth form.

FIG. 37D shows that both sides 776 and 778 of the tooth have beenrestored to close the space and achieve normal tooth form.

FIG. 37E shows missing second bicuspid tooth 780.

FIG. 37F shows dialogue box to—781—select appropriate tooth from thedentition 782 to close space.

FIG. 37G shows tooth 784 has been automatically selected to fill space.Note that it is over contoured 785.

FIG. 37H shows that the tooth 784 form has been shaped to look moreappropriate. Note that there is a little space 786 left. This can beclosed orthodontically by moving the molar 787 forward.

FIG. 37I shows that the patient has crowding 790. Efficient resolutionof the crowding is planned by optimizing tooth movement and restorationwith veneers.

FIG. 37J shows that the teeth 792 and 794 have been movedorthodontically to help resolve some of the crowding.

FIG. 37K shows that all teeth 793 have been moved orthodontically tohelp resolve some of the crowding.

FIG. 37L shows the selection of automatic veneer build up 795 to resolvethe crowding completely and to provide better shape and form. 3-Dsimulation planning has allowed the optimal planning of care from bothrestorative and orthodontic perspectives with minimal tooth movement anddestruction of tooth.

FIG. 37M shows fractured incisal edge 796. Choice of either restoringthe fracture by using a template tooth, or a similar non-fracturedtooth, or choosing either the mesial or distal side of any normal shapedtooth to restore edges or line angles.

FIG. 37N shows fractured incisal edge restored in normal shape and form798.

FIG. 38A shows composite image of craniofacial complex by combiningcraniofacial bones 800, tooth crown and roots 802 from CBCT, gingivaltissue 804 from surface scanning, and facial soft tissue 806 from 2-Dphotographs and tooth with roots; and the lower jaw and its teethregistered to the upper jaw and teeth using an intraoral biteregistration scan.

FIG. 38B shows composite image of craniofacial complex by combiningcraniofacial bones 810, tooth crown 812 and roots from CBCT, gingivaltissue 814 from surface scanning, and facial soft tissue from 2-Dphotographs and tooth with roots; and the lower jaw and its teethregistered to the upper jaw and teeth using an intraoral biteregistration scan. The figure also shows functional movements such asmouth opening.

FIG. 38C shows volumetric 3-D image of teeth and gingival tissue 814 inrelation to the patient's face in order to assess and plan foraesthetics.

FIG. 39A shows the dental complex comprising the roots—900—and thecrowns 902, which can be oriented to user defined reference planes. Forexample reference from the frontal perspective has been defined by aline 904 between the pupils of the eyes and a line 906 perpendicular tothis line 904. Similarly, user defined reference plane. A secondreference plane (not shown)_has been chosen from the lateralperspective. The combination of frontal reference plane and lateralreference plane, with a common coordinate system with respect to themodel and the face allows for correct orientation in 3-D space. The faceand its accompanying structures 908 can also be oriented by the userfrom any perspective including frontal and lateral.

FIG. 39B shows a side view of the dental complex showing bones 920,teeth 922 and a reference plane 924.

FIG. 39C shows different orientations of the face 930 of the patient.

The preferred embodiment of the invention combines volume scan data withsurface scan data to get the benefit of both and compensate forweaknesses of each.

The advantages of volume scan data are (i). acquisition of invisibledata (CBCT & MRI) such as (a) roots, bone, condile, Airways; whereas theadvantages of the surface scan data are high accuracy and resolution onvisible surfaces.

The goal of the invention is to obtain (a.) high accuracy representationof visible areas, especially small features on teeth, (b) representationof gingival, (c) representation of tooth roots, (d) representation ofbones, (e) representation of condole, and (f) representation ofbrackets, all in very high precision 3-D modeling by combining surfacescan data with the volume scan data.

In summary, method and workstation for generating three dimensionaldigital or virtual model of the dentition and surrounding anatomy of apatient from surface scan data and volume scan data are disclosed.Surface scans of a patient's dentition are obtained using in-vivoscanning or other types of scanning such as scanning an impression ofthe patient's dentition or scanning a physical model of the patient'sdentition. Volume scan data of the patient's dentition are obtainedusing Cone Beam Computed Tomography (CBCT) or Magnetic ResonanceTomography (MRI) imaging equipment. By registering the surface scan datawith the volume scan data three dimensional models of a patient'sdentition and surrounding anatomy including roots, bones, soft tissues,airways, etc. are obtained.

parts First and foremost the essence of the patent is the ability tocapture images from various soy es CBCT, cat, MRI fmri .ultrasound,still photos, intraoral scanners and videos both static and dynamicplease refrain from calling CBCT invasive.

With these images a composite structure of the face can be constructeddynamic or static We can also track function or jaw movement andsimulate the functional movements eg smile movement of the lower jawe.t.c.

Most Importantly from the CBCT we can extract root, and bone data andsoft tissue and if there is any attached appliance such as orthodonticbrackets without taking multiple images in one sweep and process eachcomponent to create separate objects to use for treatment planning andcustomized appliance selection or design and manufacture. Thesimulations allow us to reposition any component bone soft tissue toothwith roots with respect to each other in a measured way and chosenreference planes. Furthermore we can change and restore both the shapeand form of any of the structures to modify the appearance of any ofthese structures eg tooth shape or gum tissue etc. These changes both interms of position and shape can be driven by external data, e.g.,templates or normative data or internal data the non-affected side ofthe patient or combination thereof.

We can also replace or remove any of the structures to achieve thedesired goal, e.g., Implants or extraction

In essence we can reposition restore replace or remove any of the saidobjects. The codependency of movement of one object and its effect onanother can also be simulated for all three tissue types, e.g., when thetooth moves how does it affect the gum soft tissue when the tooth moveswhere does the root move in reference to the bone or how does the bonechange how does the face change when the bones move. As a result alltypes of planning can be executed in by various professions in aninteractive manner asynchronously or synchronously these may include theorthodontist, maxillofavcial surgeon, prosthodontist, perodontist,restorative dentist. Also function can be simulated or modeled basedupon captured data to achieve the desired goals, e.g., the teeth withtheir roots can be appropriately positioned in the bone to withstand thestresses of jaw movement or that the position of the jaw joint ie thecondyle is in harmony with the position of the teeth to prevent anysource of dysfunction all these simulation involve natural anatomicalstructures being affected in 3d space with volumetric data or incombination with 2d data when appropriate

The treatment plan can be used to generate any kind of dentalorthodontic restorative prosthodontic or surgical device may it tissueborne dental borne or osseous borne or any combination singularly inserial or in parallel some devices e.g., brackets indirect bonding traysstents fixation plates screws implants surgical splints crowns implantsprosthetic devices dentures or prosthetic parts to replace or restoreany tissue manufacture can be done by stereo-lithography milling orbuild up processes furthermore this data can be used to drivenavigational systems for performing any procedure and simulations can beused to train and build skills or examine proficiency another examplethe output can be used to drive robots to perform procedures and lastlythe treatment plan can be printed to provide a solid modelrepresentation.

Registration can be made at three levels. One is the orientation of theface, secondly the orientation of any component soft tissue to teeth orbone by using appropriate reference planes that are user defined oranatomically defined, and finally the bite registration by taking theintramural scan and registering the CBCT to it or a scan of the biteregistration material, e.g., wax and registering to it Please make surewe do not fall short on describing planning care with true anatomicalstructures and such as roots and the freedom to plan around and with anychosen object. It is very important that you clarify that we do not fusea model of the dentition into the crank facial structure we can captureall in one shot and extract individual features again roots and softtissue etc. In the specs we have to make this very clear you appear tosuggest taking a model of the debtyion with an intraoral scanner andfusing it with the CBCT this is not the main thrust of the app it is thefact that we can capture the dental and osseous and soft tissue as oneand segregate them individual components for planning.

The optimization of the treatment plan can be accomplished by usingdifferent approaches, e.g., correcting crowding by minimizing toothmovement and planning veneers or minimizing tooth preparation for veneerconstruction by positioning the teeth appropriately. This can be saidfor any structure and the decision can be driven by the patients need,time constraints, cost risk benefit, skill of operator, etc.

The process to extract roots based on well known concepts is as follows:

1. Interactively, select a good threshold value which captures the roots

2. Extract the surface or surfaces identified in step 1, representingthem as triangles

3. Interactively, apply any needed clean up—remove unwanted data andmerge any needed, disconnected fragments

4. Interactively, separate the data (triangles) into separate,individual tooth objects

5. Interactively, apply any needed clean up to each tooth object

The bone surface can be extracted similarly, as follows:

1. Interactively, select a good threshold value which captures themandible, maxilla, and potentially, the teeth

2. Extract the surface or surfaces identified in step 1, representingthem as triangles

3. Interactively, apply any needed clean up—remove unwanted data andmerge any needed, disconnected fragments

4. Using boolean (set) operators, subtract the tooth objects from theextracted surfaces

5. Interactively, separate the mandible from the maxilla by removing anyedges and triangles which connect one to the other.

This process can be executed in any of various available tools that canread a CBCT data set (DICOM) and find an iso-surface based on athreshold value. One such software tool is Amira.

While presently preferred embodiments of the invention have beendescribed for purposes of illustration of the best mode contemplated bythe inventors for practicing the invention, wide variation from thedetails described herein is foreseen without departure from the spiritand scope of the invention. This true spirit and scope is to bedetermined by reference to the appended claims. The term “bend”, as usedin the claims, is interpreted to mean either a simple translationmovement of the work-piece in one direction or a twist (rotation) of thework-piece, unless the context clearly indicates otherwise.

1-58. (canceled)
 59. A method of registering bite of the upper jaw andthe lower jaw of a patient, with the accompanying teeth, comprising thesteps of: (a) capturing three dimensional volumetric scan of the craniofacial and dentofacial complex of said patient; (b) obtaining an in-vivoand/or invitro bite scan of said patient's dentition using surfacescanning; (c) registering said three dimensional volumetric scan of thecranio facial and dentofacial complex to said patient's said in-vivoand/or invitro bite scan of the dentition obtained from surfacescanning. 60-63. (canceled)